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(54) Innprovements in image stabilisation 

(57) An image stabilisation method for substantially 
reducing the effects of camera shake comprises the steps 
of 

(i) capturing a first frame representative of the image 
captured by the camera at a first time, 

(ii) storing the first frame in a frame store, 

(iii) capturing a second frame representative of the image 
captured by the camera at a second time, 

(iv) measuring an image shift appertaining to a shift of the 
image captured at the second time with respect to the 
Image captured at the first time, and 

(v) forming a composite frame by over writing parts of the 
first frame stored In the frame store with common parts of 
the second frame in accordance with the measured image 
shift, thereby ensuring that the image represented by the 
composite frame will appear stable when viewed. 
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At least one drawing originally filed was informal and the print reproduced here is taken from a later filed formal copy. 
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IMPROVEMENTS IN OR RELATING TO 
IMAGE STABILISATION 

The present invention relates to methods for stabilising 
images captured by cameras. 

Ftirthermore, the present invention relates to apparatus for 
providing a camera with a means for stabilising an image. 

Images captured by video cameras and the like, suffer from 
a problem known as camera shake, wherein vibrations induce 
rotation and movement of the video camera, thereby causing a 
captured image to shake. In particular, but not exclusively, 
camera shake is a problem for hand held cameras and for cameras 
mounted on non-rigid bodies, vs^ere camera shake is inherent. 

Known systems for reducing the effects of camera shake 
include apparatus for electronicaUy manipulating an image 
captured by the camera on a display of the camera such that the 
viewer sees a stable image despite instability in the im^e 
captured by the camera. To this end, known electronic 
stabilisation systems operate to measure an image shift of the 
image captured by the camera with respect to sensors within the 
camera. An image shift measurement may be made in various 
ways, including using sensors to measure an angular movement of 
the camera. Furthermore, with knowledge of the focal length of 
the camera lens, a corresponding image shift on a camera sensor 
may be calculated. Alternatively, featiures of an image captured 
by the camera can be tracked using image processing techniques. 
Once image shift has been measured, prior art stabilisation 



3/8/2007, EAST Version: 2.1.0.14 



systems thereafter operate to display each of a plurality of frames 
of the image shifted in accordance with the measured image shift 
to compensate for the shift in the image caused by camera shake, 
thereby providing a viewer with a displayed scene v>^erein at 
least some part of the image appears stable. 

It is hereby stated for clarity that an image formed by a 
camera is a representation of a scene within the field of view of 
the camera in accordance with light emanating from the scene and 
passing into the camera, whereas a frame is a signal or an amoimt 
of data representative of the image captured at a particular time. 

The effects of known electronic image stabilisation systems 
is illustrated in Figure 1. hi Figure 1(a) an example of a displayed 
unage effected by camera shake is presented, wherein a plurality 
of superimposed frames of an inu^e representative of a scene of a 
car 1, and a lamp post 2, are shown to move to various positions 
within the displayed image by a display means. The superposition 
of frames representative of the image of the scene captured at 
different positions of the camera, has an effect of obscuring visual 
information of the scene of the car 1 and lamp post 2. Figure 1(a) 
therefore presents a representation of an unstable image of a car 
1 and a lamp post 2. 

In Figure 1(b) tiiere is presented a representation of a 
displayed image after the known electronic stabilisation system 
has operated on the unstable hnage presented in Figure 1(a). to 
Figure 1(b), the stabilised image produced by the electronic 
stabilisation means may now be seen to be centrally located and 
stable within the display. The electronic stabilisation means 



3/8/2007, EAST Version: 2.1.0.14 



-3- 

operates to shift each frame representing the image of the car 1 
and lamp post 2, such that the car 1, and lamp post 2, appear to be 
stable in the display. However, as the stabilisation process - 
operates to shift the entire image of the car 1 and lamp post 2, 
part of the display image will not contain any displayed 
information and will therefore appear blank. Furthermore, as 
indicated in Figure 1(b), solid lines 3, 4, 5, 6, 7, 8, which are 
representative of the edges of the frames of the displayed image, 
the edge of the image will be continuously be moving as the 
electronic stabilisation means operates to present a stable image 
of the car 1 and lamp post 2 within the scene. 

In practice, the known electronic stabilisation system, 
hereinbefore described, suffers a disadvantage in that although 
the stabilisation system operates to stabilise an image within the 
centre of a display, the peripheral region of the displayed image 
will be continuously moving, which is distracting to the viewer. 
To alleviate this problem known systems operate to blank out or 
crop the moving part of the image, thereby providing a stable 
representation of the scene within a centre portion of the captured 
images. Thereafter, the cropped image is expanded to fill a 
display, such that the blanked border is removed. Figure 1(c) 
therefore provides a representation of a cropped image wherein 
the edge of the scene 9, has been blanked off, whereas Figure 1(d) 
presents the stabilised image of the scene 10, expanded to fill the 
display. The known prior art image stabilisers therefore suffer a 
disadvantage in that although a stable image is produced, the 
effective image area of a scene provided by the camera in 
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combination with the image stabiliser is substantially reduced. 
More especially, when viewing an object of interest, the electronic 
image stabiliser operates to stabilise an image of the object but 
peripheral information at the regions of the object appertaining to 
contextual information will be lost, thereby greatly reducing the 
value of the image. Therefore, to produce an image of the same 
scene, a camera must be provided with a shorter focal length 
vviiich has a further disadvantage that the image will suffer a 
reduction in spatial resolution. 

It is therefore an object of the present invention to provide a 
inethod of stabilising an image captured by a camera, wherein the 
disadvantages of the aforementioned prior art are substantiaUy 
reduced. 

It is a further object of the present invention to provide an 
image stabiliser for a camera wherein the disadvantages of known 
stabilisation systems are substantially mitigated. 

According to the present invention there is provided a 
method of stabilising images captured by a camera, comprising 
steps of; 

(i) capturing a first frame representative of the image captured 
by the camera at a first time, 

(ii) storing the first frame in a frame store, 

(iii) capturing a second frame representative of the image 
captured by the camera at a second time, 

(iv) measuring an image shift appertai n ing to a shift of the image 
captured at the second time with respect to the image captured at 
the first time, and 
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(v) forming a composite frame by over writing parts of the first 
frame stored in the frame store with common parts of the second 
frame in accordance with the measured image shift, thereby 
providing a substantially stable image, represented by the 
composite frame, when viewed. 

By over writing the first frame with common parts from the 
second fi:ame, any contextual information present in peripheral 
regions of the image will be retained when displaying the over 
written composite frame. Thereafter, when further images are 
captured at subsequent times and frames representative of these 
captured images are used to over write the information in the 
frame store, such that common parts only are over written^ the 
composite frame produced wiU be provided with peripheral 
contextual information generated from a plurality of captured 
images each of which was captured at a different time, and which 
may correspond to a different position of the camera as the 
cameria has moved in accordance with camera shake. The method 
of stabilising an image therefore provides a stabiUsed image of an 
object with contextual information at the periphery of a scene in 
which the object lies, witiiout a need for increasing the focal 
lengtii of the camera, and retains all information captured by xhe 
camera. 

A displayed image produced in accordance with the method 
of stabilising images produced by a camera, therefore consists of a 
last frame of an image captured by the camera, super imposed on 
a mosaic of parts of previous frames of images. As a result 
camera shake may result in some areas of a scene not being 
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refreshed for many frames, which may further result in a 
situation wherein parts of the displayed composite frame are no 
longer consistent with a latest image captured by the camera. 
This situation may occur when a scene is changing rapidly, for 
example, where cars or people are moving into or out of the scene. 

The method of stabilising an image may further comprise a 
step of high-pass fUtering the measured image shift before step 

(V). 

To reduce the impact of inconsistencies in the composite 
image, the measured image shift may be high-pass filtered so that 
the siabiUsation method does not act to remove camera shake at 
frequencies below a predetermined cut-off frequency of the filter. 
As a result, the stabiUsed image will appear to move in response 
to camera movements representative of image shift frequencies 
below the cut-off. If die cut-off frequency is low enough diis 
slow movement of the image will not be distracting. If it is 
required that the selected image featiire be stable for shake 
frequencies below the cut-off it wiU be necessary to ensure that 
the camera follows the selected object. At low frequencies this 
will be easy to achieve. 

According to an aspect of the present invention tiiere is 
provided an image stabiUser for stabilising a displayed image 
captured by a camera, the said camera having means for 
representing the image as a plurality of frames, each of which 
frames being representative of die image capnu-ed by the camera 
at different times, the image stabiliser comprising 
a frame store for storing data representative of the frames. 
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an image shift measurer which operates to measure an image shift 
appertaining to a shift of the image from one time to another time, 
and a control unit connected to the image shift measurer and to 
the frame store which operates to overwrite parts of the frame 
store which parts are common to the captured image after the 
ca«)tured image has been shifted in accordance with the image 
shift. 

One embodiment of the present invention will now be 
described by way of example only, widi reference to the 
accompanying drawing, wherein. 

Figure 2 is a schematic and somewhat conceptual block 
diagram of an image stabilising system. 

In Figure 2 a video camera 20 is positioned such that a 
scene of interest is within a field of view 26, bounded by soUd 
lines 22 and 24. The video camera 20 may be positioned on an 
observation platform high above the ground in order that the 
scene of interest falls within the field of view 26. As such the 
observation platform may suffer from vibrations caused by 
actions of wind and other vibrations, thereby causing shaking in 

the camera 20. 

The camera 20 operates to generate frames of information 
representative of an image captured by the camera 20 of the 
scene of interest within the field of view 26. Frames that are 
generated in accordance with analogue signals are communicated 
from the camera 20 to an analogue to digital converter 28, which 
operates to generate digital representations of the analogue 
fi^es. Digital representations of the analogue frames are 
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thereafter fed to a frame store 30, under control of a control unit 
32. The control unit 32 is comprised of an image shift measurer 
34 and a control processor 36. The image shift measurer 34 
operates to measure a shift in an image captured at a particular 
time represented as a digital frame generated by the analogue to 
digital converter hi combhiation with the camera 20, with respect 
to an image captured at an earlier tune. The control processor 36, 
operates in accordance with the unage shift measurer to write the 
frames representative of images captured by the camera 20 ui a 
frame store 30. Each digital frame fed from the analogue to digital 
converter 28 is thereby written uito the frame store 30 at a 
position whereui an image or object of interest widiin an image 
captured by the camera 20 appears to be stable and vsiierein 
common parts of an image are over written by common parts of a 
subsequent image, thereby providing that the composite image 
comprises common parts and composite parts generated from 
frames representative of earUer images. The composite frame 
stored in the frame store 30, is thereafter fed to an unage display 
means 38, which thereafter communicates the image to a screen 
40. 

As illustrated in Figure 2, die displayed image on the screen 
40, corresponding to the composite frame is comprised of the 
common parts 42 and the composite parts 44. The unage 
displayed on the screen 40 is represented as a stable image with 
contextual information from the camera 20 at a periphery of the 
image 42 bemg comprised of parts from earUer frames. In tiiis 
case, the object of hiterest is the car 1 and the lamp post 2 vs^ch 
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appear substantiaUy in the centre of the screen 40. Objects 
surrounding the car 1 and lamp post 2 are thereby presented at 
the edges and periphery of the scene which are generated by 
frames captured from earlier images. 

Where the scene displayed by the image stabiUsing system, 
shown in Figure 2, is changing rapidly, objects will be moving into 
and out of the scene, hi this case a situation can occur wherein 
inconsistencies appear in the image displayed by the display 
means 40, wherein objects remain within the displayed image 
after diey have in reaUty left the scene. To mitigate 
inconsistencies in the displayed image the output of the control 
processor 36 is high-pass filtered so that die stabiUsation method 
does not act to remove camera shake at frequencies below a 
predetermined cut-off frequency of the filter. If this is 
implemented the shifted image will not remain in one area of the 
screen for a long period of time. Instead the shifted image wiU 
slowly move back to the centre and the inconsistencies in the 
image will be overwritten. The cut-off firequency is set so that 
changes in the scene have a statistically low probability of 
generating inconsistencies in the displayed image. 

To effect panning (i.e. deliberate movement of die image) an 
image shift appertaining to movement of the image ui accordance 
with the intended panning could be removed from die measured 
image shift so tiiat die stabiUsed image moves smootiily, as 
intended. To tiiis end, a panning signal representative of 
movement of die camera in correspondence widi intended 
movement of die image, may be provided by a camera control 
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system. Such a camera control system would operate to provide 
remotely controlled movement of the camera in accordance with 
control signals generated by an operator, which control signals 
could furthermore be used to generate the panning signal. 
Alternatively the panning signal could be generated by a 
movement sensor, in response to physically sensed movement of 
the camera. By subtracting the panning signal from the image 
shift signal a resultant shift signal would be used by the control 
unit to shift the image in the frame store in sympathy with the 
intended panning. 

The camera 20 shown in Figure 2, may be a video camera, or 
it may be an infra red camera generating infra red images. 
Furthermore, the camera 20 may be provided with a means for 
generating images in accordance with microwave signals received 
thereby. 

As will be appreciated by those skilled in the art, various 
modifications may be made to the stabilisation method and 
apparatus hereinbefore described without departing from the 
scope of the present invention. In particular, effects of jitter 
caused when a camera is tracking (panning or tilting) may also be 
mitigated using tiiis apparatus. In this case die image stabiUsing 
system operates to arrange for the images captiired ui accordance 
with a tracking motion of die camera to appear to be smooth and 
relatively still, whilst tiracking is being performed. Furthermore, 
the stabiUsation system can be appUed to cameras to stabilise 
images for security purposes or for broadcast appUcations wherein 
die camera is used for a sporting event and is mounted on, for 
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example, a grandstand or on a vehicle v\*dch is subject to 
vibration. 
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CLAIMS 

1. A method of stabilising an image captured by a camera, 

comprises the steps of; 

(i) capturing a first frame representative of the image captured 
by the camera at a first time, 

(ii) storing the first frame in a frame store, 

(iii) capturing a second frame representative of the image 
captured by the camera at a second time, 

(iv) measuring an image shift appertaining to a shift of the image 
captured at the second time with respect to the image captured at 
the first time, and 

(v) forming a composite frame by over writing parts of the first 
frame stored in the frame store with common parts of the second 
ft^e in accordance with the measured image shift, thereby 
providing a substantially stable image, represented by the 
composite frame, when viewed. 

2. A method of stabilising an image as claimed in Claim 1, 
wherein the measured image shift if filtered before step (v). 

3. A method of stabilising an image as claimed in Claim 2, 
wherein the measured image shift is high-pass filtered before step 

(V). 

4. A method of stabilising an image as claimed in any 
preceding claim, further including the steps of generating a 
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panning signal representative of a shift of the image in 
correspondence with a panning movement of the camera, and 
subtracting the panning signal from the image shift before step 

(V). 

5. An image stabiliser for stabilising a displayed image 
captured by a camera, the said camera having means for 
representing the image as a plurality of frames, each of wiiich 
frames being representative of the image captured by the camera 
at a particular time, the image stabiliser comprises 

a frame store for storing data representative of the said frames, 
an image shift measurer which operates to meaisure an image shift 
appertaining to a shift of the image from one time to another time, 
and a control unit connected to the image, shift measurer and to 
the frame store which operates to overwrite parts of the frame 
store which parts are common to the captured image after the 
captured image has been shifted in accordance with the image 
shift 

6. An image stabiUser as claimed in Claim 5, wherein the image 
shift measurer comprises at least one sensor which operates to 
measure angular movement of the camera. 

7. An image stabiliser as claimed in Claim 6, wherein the said 
at least one sensor operates to measure the image shift in 
accordance with a focal length of the camera lens. 
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8. An image stabiliser as claimed in Claim 6 or 7, wherein the 
said image shift measured by the image shift measurer is also 
representative of a rotation of the image. 

9. An image stabiliser as claimed in any one of the Claims 5 to 

8, further comprising an analogue to digital converter being 
connected to the camera which operates to convert analogue 
frames generated by the camera into digital frames, which digital 
frames are stored in the frame store. 

10. An image stabiliser as claimed in any one of the Claims 5 to 

9, further comprising a display means v/tdch operates to display 
images appertaining to the frames stored in the frame store in 
communication therewith. 

11. An image stabiliser as claimed in any one of the Claims 5 to 

10, wherein the control unit further comprises a comparator 
which operates to compare the image shift with a predetermined 
threshold and wherein the control unit operates to refresh the 
store with the frame captured at the said another time, where the 
unage shift exceeds the predetermined threshold. 

12. An image stabiliser as claimed in any one of Claims 5 to 10, 
further includes a filter behig connected to the image shift 
measurer which filter operates to filter the image shift before 
being communicated to the control unit. 
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IS. An image stabiliser as claimed in Claim 12, wherein the filter 
is a high-pass filter. 

14. An image stabiliser as claimed in any preceding claim, 
v>^erein the control unit operates to subtract a panning signal 
representative of movement of the image in correspondence with 
a panning movement of the camera from the image shift. 

15. An image stabiliser as claimed in Claim 14, wherein the 
panning signal is generated from control signals generated by a 
camera control system, which camera control system operates to 
remotely move the said camera in response to the said control 
signals. 

16. An image stabiliser as claimed in Claim 14, wherein the 
panning signal is generated by a movement sensor in accordance 
with a sensed movement of the said camera. 

17. An image stabiliser as claimed in any one of the Claims 5 to 
16, wherein the image is captured by the camera in accordance 
with microwave, millimetre or ultrasonic signals received thereby. 

18. An image stabiliser as claimed in any one of the Claims 5 to 
16, wherein the image is captured by the camera in accordance 
with infra red radiation received thereby. 
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19. A camera including an image stabiliser as claimed in any one 
of the Claims 5 to 18. 

20. An image stabiliser as hereinbefore described with reference 
to Figure 2. 
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